IAES International Journal of Artificial Intelligence (IJ-AI) 
Vol. 12, No. 4, December 2023, pp. 1727~1734 
ISSN: 2252-8938, DOI: 10.1159 L/ijai.v12.i4.pp1727-1734 O 1727 


Analysis of machine repair time prediction using machine 
learning at one of leading footwear manufacturers in Indonesia 


Yulio Agefa Purmala, Sumarsono Sudarto 


Department of Industrial Engineering, Universitas Mercu Buana, Jakarta, Indonesia 


Article Info 


ABSTRACT 


Article history: 


Received Aug 8, 2022 
Revised Jan 31, 2023 
Accepted Mar 10, 2023 


Keywords: 


Classification 
Evaluation metrics 
Machine breakdown 
Machine learning 
Repair time 


Machine breakdowns in the production line mostly finish in more than 18 
minutes, since the machine that needs repair more time is done on the 
production line, not in the machine warehouse. Historical machine 
breakdown data is digitally recorded through the Andon system, but it is still 
not being used adequately to aid decision-making. This research introduces 
an analysis of historical machine breakdown data to provide predictions of 
repair time intervals with a focus on finding the best algorithm accuracy. 
The research method uses machine learning techniques with a classification 
model. There are five algorithms used: logistic regression (LR), naive bayes 
(NB), k-nearest neighbor (KNN), support vector machine (SVM), and 
random forest (RF). The results of this study prove that historical machine 
breakdown data can be optimized to predict machine repair time intervals in 
the production line. The accuracy of LR algorithm is slightly better than the 
other algorithms. Based on the receiver operating characteristic—-area under 


curve (ROC-AUC) performance evaluation metric, the quality value of the 
accuracy of LR model is satisfied with a percentage of 69% with a 
difference of 0.5% between the train and test data. 
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1. INTRODUCTION 

Based on export data released by the Direktorat Jenderal Bea dan Cukai Indonesia (Indonesian 
Directorate General of Customs and Excise) taken from export declaration of goods documents in 2020, the 
value of Indonesia's exports decreased by 2.68% from the previous year, reflecting the impact of COVID-19. 
The decline in the value of Indonesia's exports in 2020 was caused by a 30.01% decline in commodity 
exports of oil and gas, as well as a 0.61% decline in non-oil and gas exports [1]. The role of Indonesia's oil 
and gas and non-oil exports has shifted. Non-oil and gas exports accounted for 94.94% of total exports in 
2020, an increase of 1.97% from 2019. One of the non-oil and gas commodity sectors, namely the 
manufacturing industry sector, has the largest contribution to total exports. Its contribution will be 80.33% by 
2020. One of the export commodities that experienced an increase was the footwear sector, especially 
sporting goods. Export value increased by 31.09% in 2020 compared to the previous year. Indonesia is 
among the top four countries with the highest number of footwear exports, still behind China, India, and 
Vietnam [2]. 

The footwear industry is a labor-intensive industry [2]. Indonesia has about 18,687 footwear 
business units with a workforce of 795,000 manpower, which are then followed by the required number of 
machines. This is to meet high demand for shoes, but still limited in technology and capital, so that Indonesia 
can compete with other exporting countries, especially China and Vietnam. If the focus is on the needs of the 
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machine, then the function or type of machine used is also very diverse, adjusting to each process and model 
of footwear that must be done. This research will focus on the cutting, stitching, and assembly (CSA) process 
because most processes and stages of manufacturing and assembling components are carried out in that 
process. Based on data that has been collected in one of the leading footwear manufacturing industries in 
Tangerang, Indonesia, it takes about 10,633 machines in the CSA process, which consists of 106 types of 
machines and/or 375 series of machines that must be managed properly. With many machines that must be 
managed, the handling and control of machines must also be carried out very well. The effectiveness of 
several machines in supporting production activities to fulfill exports is closely related to the availability and 
reliability of machines. Technicians always try to ensure machine availability and reliability are maintained 
through regular maintenance activities [3], but due to the large number of machines used, machines 
sometimes break down before maintenance time is completed [4], [5]. 

In 2021, the number of machine breakdowns that occur in production lines is very high. Currently, 
all machine breakdowns that occur in the CSA process are well recorded using the Andon system. Besides 
being used as a tool to call technicians, Andon has also been used as a tool for storing historical information 
on machine breakdowns [6]. Five machines in particular had a high breakdown rate and made the biggest 
contribution to the overall breakdown, namely: IN postbed stitching machine, 2N postbed stitching machine, 
computer stitching machine-Medium, skiving machine, and computer stitching machine-Large. In addition to 
availability, the reliability of the machine must also be controlled properly. The average time for repairing a 
machine breakdown from the five machines above is 27.6 minutes. 

The export process will be disrupted if the production process has problems included because of 
machine breakdown time, so machines that are broken for more than 20 minutes must be worked in the 
machine warehouse. The next problem is that the technicians cannot predict it. In addition to the uneven 
distribution of technician expertise, this is also because they still tend to rely on experience and intuition in 
estimating repair time intervals. Historical data on machine breakdown is recorded digitally through the 
Andon system, but the data is still not properly utilized to assist decision making. The aim of this research is 
to analyze historical data on machine breakdown to provide predictions of time intervals for repairing 
machines with a focus on finding the best algorithm accuracy using a machine learning approach. 


2. METHOD 

This study uses the cross-industry standard process for data mining (CRISP-DM) model process for 
a universal data analysis approach [7], [8]. Several supervised machine learning classification method 
algorithms were chosen to get the best accuracy value [9], [10]. The algorithms are logistic regression (LR) 
[11], [12], naive bayes (NB) [13], [14], random forest (RF) [15], [16], k-nearest neighbor (KNN) [17], [18], 
and support vector machine (SVM) [19], [20]. Selected five classifications supervised machine learning 
based on each algorithm have on different dimension metrics there are parametric-simple for LR and NB, 
parametric-complex for SVM, non-parametric-simple for KNN, and non-parametic-complex for RF. Several 
dimensions of the variables involved in this study include reports of machine breakdown, such as repair time, 
response time, machine model or type, building location, type of breakdown, causes of breakdown, and repair 
solutions. Variable dimensions of assets, such as asset number, machine age, machine price, machine arrival 
date, and machine ownership status. Machine replacement, such as the location of the old and new buildings. 
Employee variables such as position and years of work experience. The Framework in this study describes 
how the concept of data utilization using machine learning methods [21], [22]. An overview of the 
framework can be seen in Figure 1, how data sources get and which variable is used, next go to how data are 
transformed with several data modeling, performance analysis, and finding the best model to get the output of 
the prediction accuracy. 
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Figure 1. Research framework 
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There are two evaluation metric methods that will be used: confusion metrics [23] and receiver 
operating characteristic - area under curve (ROC-AUC) [24]. Confusion metrics is an evaluation model to 
find the values of accuracy, precision, recall, and F1 score by looking at the probability value between actual 
value and predicted value at one threshold. Meanwhile, ROC-AUC evaluates all possible performances at all 
thresholds that are under the ROC curve area. At the end of this study, ROC-AUC will be used as the main 
evaluation [25], while the evaluation of confusion metrics will be used as a supporting evaluation [26]. To be 
more effective and time efficient all machine learning methods are processed by platform orange data mining 
and do some setup parameters in some algorithms. LR set up on a ridge with a coefficient score is 11, RF 
setup with 10 number of trees, SVM setup with cost is 10 and regression loss epsilon is 0.10, and KNN setup 
with 10 number of neighbor and metric euclidean. 

In addition to using a statistical approach, another way of determining variables in addition to using 
a statistical approach is through a domain expertise approach, with direct brainstorming to the engineering 
department leaders starting from the supervisory level to the manager level, through focus group discussions 
(FGD) [27]. After several independent variables have been determined, a correlation test is carried out with 
the dependent variable, that is, repair time interval, using the chi-squared test [28]. Prior to modeling, data 
will be divided into two with a ratio of 70:30, 70% as training data and 30% as test data. The flow chart of 


this research is shown in Figure 2. 
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Figure 2. Research flow chart 


3. RESULTS AND DISCUSSION 

Several data sets are already recorded digitally, but there are also those that use worksheets in data 
collection and require a data transformation process from manual to digital with a computerized system. The 
following is the flow of data used in this research, as shown in Figure 3. Machine breakdown in Andon, spare 
parts transactions, and equipment assets data are recorded digitally and stored directly on the server, work 
order forms need to transform from paper to computerize before store on the server. The dataset in this 
research is exported directly from the internal server and cloud server. The data used for modeling analysis is 
in the (.csv) format to make analysis easier because this format can be used for many systems and is compact 
and straightforward [29]. Data that has been collected is then carried out a thorough depiction of data for 
each variable, as well as checking whether there is empty or missing data contained in each data set. 
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Figure 3. Diagram of data flow 


3.1. Data preparation 

Data that has been thoroughly prepared, inspected, and described is still very messy and needs to be 
preprocessed so that it is ready for use in machine learning modeling. Based on the initial description of each 
data set as shown in Figure 4, where there is missing data, which will then be processed to handle the missing 
data. Missing data that is still less than 5% does not need to be removed; it only needs to be imputationed by 
replacing it with the average or median value for numeric data types, and it is replaced based on the 
frequency with which it appears for categorical data types [30]. The variable target in this research is the 
repair time interval. Whether the required repair time is less than 18 minutes or not, the determination of the 
18 minutes repair time standard is based on the machine breakdown history during 2021, where the average 
repair time is 24 minutes and the median time is 18 minutes, as shown in Figure 5. 
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Figure 4. Pre-processing data results 
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Figure 5. Machine breakdown time variable in statistics 


Based on the detailed analysis shown in Figure 6, the distribution of machine breakdown time is a 
positive skewed distribution. The median value is more appropriate to use for the distribution of positively 
skewed and negatively skewed data [31]. This is what underlies the determination of the machine repair time 
in this study, which is 18 minutes based on the median result of the breakdown time data. 
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Figure 6. Machine breakdown time has a positive skewed distribution 


3.2. Data modeling and analysis 

The data set is divided into two parts for modeling evaluation: training data and testing data. The 
distribution of training and test data from the total data set is 70:30. The entire data utilized for modeling 
comes from 4,163. Therefore, when employing a 70:30 ratio, 2,915 training data and 1,248 test data will be 
employed. The cross-validation approach will be used to confirm the model's performance for several 
evaluations. In this research, the modeling will be evaluated using 5-fold cross validation [32]. 

As seen in Table 1, the results of train data modeling using AUC evaluation, for LR=0.688, 
NB=0.677, RF=0.644, KNN=0.638 and SVM=0.530. Results of test data modeling results using AUC 
evaluation, for LR=0.693, NB=0.670, RF=0.673, KNN=0.626 and SVM=0.531. 


Table 1. Training and test data score comparison 

AUC Accuracy Precision Recall Fl 
Train Test Train Test Train Test Train Test Train Test 
LR 0.688 0.693 0.646 0.643 0.647 0.644 0.646 0.643 0.646 0.643 
NB 0.677 0.670 0.632 0.617 0.632 0.617 0.632 0.617 0.631 0.617 
RF 0.644 0.673 0.607 0.631 0.607 0.631 0.607 0.631 0.607 0.631 
KNN 0.638 0.626 0.600 0.696 0.603 0.596 0.600 0.596 0.597 0.596 
SVM __ 0.530 0.531 0.515 0.505 0.515 0.506 0.515 0.505 0.514 0.479 


Algorithm 


Predictions made by test data are the result of learning from training data. Machine learning 
algorithms will perform modeling based on their respective formulations and techniques and then determine 
the probability of whether to enter the classification under 18 minutes or above 18 minutes for repair time by 
considering the variables that have been determined in this study. Table 1 compares the outcomes of the 
modeling evaluation between the training and test data. From this comparison, the difference between 
training and test data is not considerably different, hence LR has reasonable modeling accuracy. The results 
of the test data modeling show that the LR, NB, RF, and KNN algorithms are in the satisfactory category, 
while the SVM algorithm is in the unsatisfactory category. The results of the confusion metrics between the 
actual and predicted data can be seen in Figure 7. 
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Figure 7. Confusion metrics of test data 


3.3. Model performance evaluation and quality model result 

This research seeks to determine the degree of modeling accuracy, using both training and test data. 
The gap between training and test data is not a serious issue if the difference is not too large, regardless of 
whether it is overfitting or underfitting. In accordance with the outcomes of machine learning modeling using 
five classification algorithms, LR is an algorithm that fits the variables in this study with the difference 
between the training data and the test data being -0.005, meaning that test data is better at 0.005 than training 
data, or at a percentage of 0.5%, according to Figure 8. 
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Figure 8. The AUC of the evaluation metrics gap between training and test data 


Based on the conclusion of data evaluation on training data and test data, for LR, NB, RF, and KNN 
algorithms in this study, the modeling quality is in the satisfactory category because the AUC value is around 
0.6 to 0.7 [24]. As for the SVM algorithm in this study, the quality of the modeling is in the unsatisfactory 
category because the AUC value is below 0.6, but the modeling is still accepted because the AUC value is 
still above 0.5. LR algorithms are the best result since the dependent variable is binary and data structure is 
simple. 


4. CONCLUSION 

This research wants to know the level of accuracy of the modeling that has been done both on the 
training data and test data. The difference between training data and test data is not a significant problem if 
the difference between the two is not too great, be it overfitting or underfitting. The results of this study 
prove that historical machine breakdown data can be optimized to predict machine repair time intervals in the 
production line within under 18 minutes. The accuracy of LR algorithm is slightly better than the other 
algorithms. Based on ROC-AUC performance evaluation metric, the quality value of the accuracy of LR 
model is satisfied with a percentage of 69% with a difference of 0.5% between the train and test data. In 
further research, the variables used can be enriched, so that the percentage of the results of the analysis of the 
resulting model will be even better. In addition, it can be developed to the implementation stage and 
integrated into existing maintenance systems to provide real-time predictions. 
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